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The development of next generation systems often carries with it the promise of 
improved performance, greater reliability, and reduced operational costs. These 
expectations arise from the use of novel designs, new materials, advanced integration and 
production technologies intended for functionality replacing the previous generation. 
However, the novelty of these nascent technologies is accompanied by lack of 
operational experience and, in many cases, no actual testing as well. Therefore some of 
the enthusiasm surrounding most new technologies may be due to inflated aspirations 
from lack of knowledge rather than actual future expectations. 

This paper proposes a design heritage approach for improved reliability 
forecasting of advanced system components. The basis of the design heritage approach is 
to relate advanced system components to similar designs currently in operation. The 
demonstrated performance of these components could then be used to forecast the 
expected performance and reliability of comparable advanced technology components. 
In this approach the greater the divergence of the advanced component designs from the 
current systems the higher the uncertainty that accompanies the associated failure 
estimates. 

Designers of advanced systems are faced with many difficult decisions. One of 
the most common and more difficult types of these decisions are those related to the 
choice between design alternatives. In the past decision-makers have found these 
decisions to be extremely difficult to make because they often involve the trade-off 
between a known performing fielded design and a promising paper design. When it 
comes to expected reliability performance the paper design always looks better because it 
is on paper and it addresses all the know failure modes of the fielded design. 

On the other hand there is a long, and sometimes very difficult road, between the 
promise of a paper design and its fulfillment; with the possibility that sometimes the 
reliability promise is not fulfilled at all. Decision makers in advanced technology areas 
have always known to discount the performance claims of a design to a degree in 
proportion to its stage of development, and at times have preferred the more mature 
design over the one of lesser maturity even with the latter promising substantially better 
performance once fielded. As with the broader measures of performance this has also 
been true for projected reliability performance. Paper estimates of potential advances in 
design reliability are to a degree uncertain in proportion to the maturity of the features 
being proposed to secure those advances. This is especially true when performance- 
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enhancing features in other areas are also planned to be part of the developmental 
program. 

Several years ago the US Air Force recognized the need to include maturity 
estimates in their development planning processes. While they saw no alternative to 
expert judgment in obtaining estimates of maturity they at least wanted to bring some 
order to that judgment. For this reason a “Technology Maturity Scale” was created, an 
example of which is presented in Figure 1 . On this scale representative graduations are 
made from one (1) to nine (9) with 1 representing the lowest maturity and 9 the highest. 
Along the scale calibrating phrases were added along with examples to allow the expert 
reviewer to establish his maturity estimates on a less subjective basis. The phrase 
correlations to the numeric scale were intentionally overlapping in recognition of the 
inherent uncertainty or “fuzziness” in the process of assignment beginning with the 
“Basic Technology or Research Level” as the least mature and with the availability of 
“Integrated System Tested Units Deployed and Operational” as the most mature. No 
design was seen as completely mature, ( 10 on the scale), because in developmental 
programs, such as the ones that the Air Force was addressing, there was always thought 
to be at least some tailoring required of even off-the-shelf designs. When the maturity of 
the designs were reviewed against this scale and maturity values assigned these values 
would be used to weight the alternatives so that designs of high performance promise, but 
low maturity, would have had their potential value to the developed program adjusted 
accordingly. In this way, although it was never mentioned, the approach attempted to 
estimate the probability that the system fielded would have the performance promised. 
Or in other words, it discounted the promised performance of immature designs during 
trade studies leading up to the selection of a final design. 
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Figure 1 : US Air Force Technology Rating Scale 

As useful as this approach was, it was limited in that it did not take into account 
the variation to be expected in maturity projections. That is, not only should the 
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projected performance of immature designs be taken into account, but also the variation 
around the expected value would be expected to be larger the more immature the design. 
The authors modified the Air Force approach to account for this expected variation by 
using a probability distribution to represent the variation around the expected value at 
each maturity level, and correspondingly increased the variation by use of increased error 
factors (EF) for more immature designs. These error factors were calculated by taking 
the ratio of the 5%ile to the median (or 50%ile) for an assumed lognormal distribution, 
and therefore represented a measure of spread in the distribution about this central value. 
These distributions and error factors were then applied as appropriate depending upon the 
maturity of each of the individual design alternatives selected to represent their expected 
delivered reliability. The overall expected reliability of the baseline design, (i.e. the 
design created by selecting the preferred lower level alternative in each instance), was 
then estimated by creating a Monte Carlo model which sampled from each lower level 
distribution thus creating a distribution representing the expectations of the entire design. 
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